Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Support multi-gpu child ops and remove CPU/GPU op splitting #28

Merged
merged 3 commits into from
Apr 2, 2024
Merged

Support multi-gpu child ops and remove CPU/GPU op splitting #28

merged 3 commits into from
Apr 2, 2024

Conversation

TaekyungHeo
Copy link
Contributor

@TaekyungHeo TaekyungHeo commented Mar 13, 2024 •
edited
Loading

Summary

  • Introduced support for associating multiple GPU child operators with a single CPU operator.
  • Added support for Tensor(c10::BFloat16) in get_data_type_size.
  • Removed the method for splitting overlapping CPU and GPU operations due to its impact on operator sequencing.
    • Observed error rate reduction from 40% to 6% in Megatron traces by eliminating splitting logic.

Test Plan

$ python tools/trace_link.py --pytorch-et-file ~/Downloads/megatron_et_0.json --kineto-file ~/Downloads/megatron_kineto_0.json --output-file ~/rank0.json &
$ python tools/trace_link.py --pytorch-et-file ~/Downloads/megatron_et_1.json --kineto-file ~/Downloads/megatron_kineto_1.json --output-file ~/rank1.json &
$ python tools/trace_link.py --pytorch-et-file ~/Downloads/megatron_et_2.json --kineto-file ~/Downloads/megatron_kineto_2.json --output-file ~/rank2.json &
$ python tools/trace_link.py --pytorch-et-file ~/Downloads/megatron_et_3.json --kineto-file ~/Downloads/megatron_kineto_3.json --output-file ~/rank3.json &
$ python tools/trace_link.py --pytorch-et-file ~/Downloads/megatron_et_4.json --kineto-file ~/Downloads/megatron_kineto_4.json --output-file ~/rank4.json &
$ python tools/trace_link.py --pytorch-et-file ~/Downloads/megatron_et_5.json --kineto-file ~/Downloads/megatron_kineto_5.json --output-file ~/rank5.json &
$ python tools/trace_link.py --pytorch-et-file ~/Downloads/megatron_et_6.json --kineto-file ~/Downloads/megatron_kineto_6.json --output-file ~/rank6.json &
$ python tools/trace_link.py --pytorch-et-file ~/Downloads/megatron_et_7.json --kineto-file ~/Downloads/megatron_kineto_7.json --output-file ~/rank7.json &

$ python3 -m chakra.et_converter.et_converter --input_type PyTorch --input_filename ~/rank0.json --output_filename ~/rank.0.et --num_dims 1 &
$ python3 -m chakra.et_converter.et_converter --input_type PyTorch --input_filename ~/rank1.json --output_filename ~/rank.1.et --num_dims 1 &
$ python3 -m chakra.et_converter.et_converter --input_type PyTorch --input_filename ~/rank2.json --output_filename ~/rank.2.et --num_dims 1 &
$ python3 -m chakra.et_converter.et_converter --input_type PyTorch --input_filename ~/rank3.json --output_filename ~/rank.3.et --num_dims 1 &
$ python3 -m chakra.et_converter.et_converter --input_type PyTorch --input_filename ~/rank4.json --output_filename ~/rank.4.et --num_dims 1 &
$ python3 -m chakra.et_converter.et_converter --input_type PyTorch --input_filename ~/rank5.json --output_filename ~/rank.5.et --num_dims 1 &
$ python3 -m chakra.et_converter.et_converter --input_type PyTorch --input_filename ~/rank6.json --output_filename ~/rank.6.et --num_dims 1 &
$ python3 -m chakra.et_converter.et_converter --input_type PyTorch --input_filename ~/rank7.json --output_filename ~/rank.7.et --num_dims 1 &
  • Profiled duration: 1345.336ms
  • Simulated duration: 1431.253ms
  • Error: 6.4%

@TaekyungHeo TaekyungHeo requested a review from a team as a code owner March 13, 2024 12:30
@github-actions GitHub Actions
Copy link

github-actions bot commented Mar 13, 2024 •
edited
Loading

MLCommons CLA bot All contributors have signed the MLCommons CLA ✍️ ✅

srinivas212
srinivas212 approved these changes Apr 2, 2024
View reviewed changes
Copy link
Contributor

@srinivas212 srinivas212 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Thanks for these improvements!

@TaekyungHeo TaekyungHeo merged commit ccbc25d into main Apr 2, 2024
7 checks passed
@TaekyungHeo TaekyungHeo deleted the bugfix branch April 2, 2024 14:32
@github-actions github-actions bot locked and limited conversation to collaborators Apr 2, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Reviewers

@srinivas212 srinivas212 srinivas212 approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@TaekyungHeo @srinivas212